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TIS  —  A  Focal  Point  for  Technology  Transfer 

Summary 

t 

Lawrence  Livermore  National  Laboratory  (LLNL)  is  using  the  Technology  Informa¬ 
tion  System  (TIS)  to  establish  a  focal  point  for  technology  transfer  in  support  of  its  Office 
of  Research  and  Technology  Applications  (LLNL/ORTA).  The  program  was  started  earlier 
this  year  in  compliance  with  the  Stevenson-Wydler  Technology  Innovation  Act  of  1980 
(PL-96-480)  and  with  corresponding  Department  of  Energy  (DOE)  Order  5800.iAtIS  is 
slated  to  become  the  computer-based  reference  point  for  the  identification  of  LLNL  pro¬ 
grams  available  for  transfer  to  the  public  domain,  as  well  as  an  effective  means  for  com¬ 
municating  with  the  respective  offices  of  other  federal  agencies:  the  Federal  Laboratory 
Consortium  (FLC),  the  Technology  Transfer  Society  (T2),  the  NASA  Industrial  Applica¬ 
tions  Centers,  and  a  selected  number  of  state  and  local  offices.  Available  resources  include 
the  online  International  Technology  Transfer  Dictionary  of  more  than  2000  experts  in  the 
field,  the  TECTRA  newsletter  and  technology  databases  maintained  by  the  School  of  Busi¬ 
ness  and  Public  Administration  at  the  California  State  University  in  Sacramento,  th?  pat¬ 
ent  database  to  more  than  2000  DOE  patents  potentially  available  for  licensing,  and  a 
growing  number  of  federal  information  centers,  such  as  DOE/RECON,  DOD/DROLS, 
DOL/SEEDIS,  NASA/RECON,  and  CIS.-  TIS  is  used  for  the  creation,  report  generation,  and 
graphic  display  of  descriptive  and  numeric  data  files,  the  execution  of  interactive  models 
calculating  the  performance  of  electric  vehicles,  electronic  mail,  interconnection  to  elec¬ 
tronic  word  processors  and  typesetters,  and  nationwide  networking.  The  system  is  fully 
self-guided,  which  also  permits  those  without  computer  experience  to  search,  extract,  and 
post-process  relevant  information  in  a  productive  and  timely  manner.  This  type  of  analy¬ 
sis  can  be  used  to  identify  the  status  and  direction  of  ongoing  government-supported 
research  and  development  (R&D)  and  to  increase  industrial  productivity  by  technology 
transfer. 


1.  Introduction 

Successful  transfer  of  advanced  technology  to  industry  is  the  main  purpose  of  government-sup¬ 
ported,  high-risk  R&D.  However,  the  time  delay  between  discovery  and  application,  as  estimated  by 
Science  and  the  American  Institute  of  Physics,  is  four  to  seven  years: 
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This  delay  has  decreased  little  during  the  past  decau.,  and  much  has  been  written  about  how  to 
improve  it  and  to  inciaase  national  productivity. 

Equally  significant  is  the  absence  of  a  national  plan  in  the  United  States  for  the  evaluation  and 
organization  of  numeric  data  and  material  properties.  Countries  less  able  to  pay  for  the  expensive  instru¬ 
mentation  and  apparatus  needed  for  innovative  research  in  science  and  technology  are  harvesting  the 
results  of  our  government-support  R&D  work  while  we,  the  contributors  of  nearly  60%  of  the  world's 
technical  literature,  oblige  our  researchers  to  read  preceding  publications  time  and  time  again,  to  make 
their  own  evaluations,  or  to  buy  compiled  and  evaluated  data  from  commercial  vendors,  a  growing 
number  of  whom  are  based  abroad.  Although  this  dilemma  was  pointed  out  in  1978  by  the  Numerical 
Data  Advisory  Board  of  the  National  Academy  of  Sciences  (NAS/NDAB)1,  direct  support  for  the  evalua¬ 
tion  of  material  properties  by  federal  agencies  to  the  Office  of  Standard  Reference  Data,  National  Bureau 
of  Standards,  has  remained  embarassingly  small. 
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Base  level  of  funding  estimated  to  prevail  in 
1978,  the  funds  required  to  catch  up  with  the 
then-existing  two-  to  five-year  backlog  of  eval¬ 
uations,  and  recommendations  for  new  work  in 
existing  and  new  centers. 
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2.  Technology  Transfer  Programs 

The  main  problem  faced  by  a  technologist  in  R&D  remains  the  same:  how  best  to  find  and  us' 
transferable  technologies  in  a  timely  manner.  Fcr  some  time,  disparate  efforts  have  been  ongoing  in  the 
United  States.  Although  these  efforts  are  to  be  united  under  the  Stevenson -Wydler  Technology  Innova¬ 
tion  Act.  unification  is  difficult  because  of  the  differing  natures  of  the  efforts,  some  of  which  are  not 
computer-based.  The  Technology  Information  System  (TIS)  at  the  Lawrence  Livermore  National  Labora¬ 
tory  (LI.NL),  discussed  ’ater  in  this  report,  provides  capabilities  by  which  such  geographically  distributed 
and  disparate  transfer  efforts  can  be  used  to  establish  an  integiated  resource  for  the  access,  searching,  and 
dissemination  of  available  technologies.  In  addition,  TIS  can  provide  effective  means  for  electronic  com¬ 
munication  among  liaison  offices  of  the  national  technology  transfer  programs. 


2.1  Stevenson-Wydler  Technology 
Innovation  Act  of  1980 

To  improve  the  situation,  Congress  mandated  that  federal  agencies  make  technology  transfer  an 
integral  part  of  their  ongoing  R&D  nork.  Public  Law  96-480  formalizes  the  implementation  and  reporting 
requirements  expected  of  federal  agencies.  It  builds  on  previous  less  formal  and  separate  programs  that 
include  the  traditional  centers  of  federal  agencies,  the  activities  of  the  Federal  Laboratory  Consortium 
(FLO,  the  Technology  Transfer  (J2)  society,  and  the  Interagency  Information  Exchange  (HE)  group  and 
establishes,  at  the  national  laboratories,  new  offices  dedicated,  on  the  working  level,  to  technology  trans¬ 
fer  The  intent  of  PL-96-480  is,  therefore,  to  make  technology  transfer  a  regular  part  of  the  federal  K&U 
effort,  which  is  expected  to  accelerate  the  diffusion  of  tax-supported  innovative  work.  Specifically,  the  law 
requires  each  federal  laboratory  to: 

1  Establish  an  Office  of  Research  and  Technology  Applications  to  work  with  potential  users. 

2.  Formally  assess  laboratory  R&D  projects  determined  to  have  potential  for  successful  applications 
beyond  the  project. 

J.  Disseminate  information  on  products,  processes,  and  services. 

4.  Cooperate  with  Department  of  Commerce  transfer  and  commercialization  programs,  the  hLl_, 
and  other  technology  transfer  organizations. 

5.  As  appropriate,  orovide  technical  assistance  for  applications. 

The  execution  of  these  required  functions  is  expected  to  be  integrated  into  R&D  activity  at  agencies 
and  laboratories.  This  implies  that  the  results  of  R&D  are  to  be  reported  not  only  to  the  public  domain  via 
the  respective  existing  information  centers  of  each  agency  and  the  National  Technical  Information  Service 
(NTIS),  but  are  to  be  highlighted  as  being  ready  and  available  for  transfer  in  succinct  documents  an  i  es 
of  each  national  laboratory.  Knowledgeable  staff  members  will  act  as  contacts  and  liaison  for  inquiries 
from  other  agencies,  universities,  industry,  and  the  general  public.  Before  we  discuss  the  ongoing  acnvities 
at  LLNL,  however,  it  is  pertinent  to  review  the  background  and  status  of  technical  reporting  an 
dissemination. 


2.2  Technical  Information  Centers 

Federal  agencies  instituted  the  collection,  indexing,  and  dissemination  of  their  R&D  results  decades 
ago.  The  largest  centers  reporting  on  work  in  science  and  technology  are  those  of  the  Department  o 
Energy  (DOE/TIC),  the  National  Aeronautics  and  Space  Administration  (NA5A/S&R),  and  the  Depart¬ 
ment  of  Defers.’  (DOD/DTIC).  Most  of  their  files  are  bibliographic  directories  that  point  to  publications  in 
the  public  domain  and  are  released  as  laboratory  reports  and  as  articles  in  the  journals  of  professional 
societies.  Jointly,  these  three  agencies  sponsor  and  report  the  results  of  the  largest  investments  in  science 
and  technology.  Their  relatives  contributions  are: 
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DOE,  DOD,  and  NASA  Create  and  Maintain  the 
Largest  Bodv  of  Technical  Information 


These  large  collections  of  bibliographic  databases  are  available  online  for  interactive  searching  with 
computer  terminals,  via  telephone  dial-up,  and  are  used  by  agency  administrators,  their  contractors,  and 
other  federal  agencies.  The  systems  are  known  respectively  as  DOE/RECON,2  NASA/RECON,3,  and 
DOD/DROLS.4  Copies  of  the  major,  unclassified  databases  of  these  systems  are  being  released  periodi¬ 
cally  to  commercial  information  vendors  who  offer  them  for  searching  to  the  general  public.  Full-text 
copies  of  relevant  reports  can  be  ordered  at  cost  from  the  National  Technical  Information  Service  (NTIS).s 
Recently,  several  federal  information  centers  hava  established  special  databases  that  highlight  the  avail¬ 
ability  of  topical  technologies  or  patents. 

However,  these  federal  and  commercial  information  centers  are  stand-alone  systems,  obliging  a  qual¬ 
ified  user  to  establish  individual  contracts  and  to  use  them  on  a  one-to-one  basis.  This  requires  knowledge 
cf  system-dependent  log-on  procedures,  telephone  numbers,  identifications,  passwords,  and  protocols. 
Since  most  of  these  systems  have  developed  independently,  the  user  must  also  be  familiar  with  different 
command  languages.  Bibliographic  citations  identified  on  any  one  of  these  systems  are  usually  viewed  on 
the  screen  of  a  computer  terminal,  or  are  printed  online.  Large  volumes  of  retrospective  citations  are 
printed  offline  at  the  remote  information  center  and  are  sent  to  the  account  holder  by  postal  mail. 

The  use  of  these  information  resources  is  tedious.  The  user  has  to  eliminate  the  unavoidable  overlap 
ot  bibliographic  citations  and  manually  inspect,  organize,  and  appraise  the  value  of  the  retrieved  rerults. 
Nevertheless,  these  technical  infor  ration  centers  perform  a  vital  funf-ti.-.r.  and  represent  the  largest  effort 
in  technology  transfer  to  date. 


2.3  NASA  Industrial  Application  Centers 

NASA  established  a  highly  successful  technology  transfer  program  in  the  1960s  through  the  NASA 
Industrial  Application  Centers.  Ten  such  centers  are  now  funded  in  part  by  NASA,  by  the  host  installa¬ 
tion,  by  federal  programs  like  the  Small  Business  Administration,  and  by  recipients  of  the  technology 
itself,  e.g.,  industry.  One  of  th  '«e  centers  is  COSMIC,  the  computer  software  transfer  center  at  the  Univer¬ 
sity  of  Georgia.  The  others  at*,  nategically  located  throughout  the  country,  servicing  seven  regions  and 
two  states. 
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This  technology  transfer  program  is  a  direct  result  of  the  Space  Act  of  1958,  which  required  NASA  to 
“provide  the  widest  practicable  and  appropriate  dissemination  of  information  of  its  activities  and  results 
thereof."  NASA's  mission,  and  therefore  NIAC's,  is  to  transfer  technological  information,  as  a  spinoff  of 
the  space  program,  to  the  private  sector  for  reuse  and  reapplication. 

The  transfer  of  information  is  carried  out  through  oni.ne  sea-ches  of  computer-based  federal  and 
commercial  databases,  augmented  by  experts  at  the  host  installations,  and  NASA  field  offices.  Input  to 
their  databases  is  indexed  from  journal  articles,  books,  newspaper  stories,  and  government  reports.  The 
databases  cover  domestic  and  international  sources  on  a  wide  range  of  topics,  including  applied  science 
and  engineering,  the  physical  sciences,  business,  life  science,  and  patents. 

NIACs  provide  specified  customized  database  searches  that  take  into  account  the  qualifications  of  the 
client  (e.g.,  range  of  dates,  journals  to  be  covered,  authors,  and  cost).  Searches  may  be  ordered  by  tele¬ 
phone  or  by  mail.  The  NIAC  staff  strives  to  respond  within  48  hours. 

Until  now,  searching  has  been  done  by  conventional  means  through  telephone  dial-up  of  individual 
commercial  and  federal  information  centers.  Succinct  results  from  the  different  sources  are  printed  online 
or,  in  the  case  of  retrospective  searches,  are  mailed  as  computer  printouts  by  the  information  center.  When 
the  results  do  not  meet  the  requestor's  needs,  the  search  must  be  repeated.  This  process  increases  user 
costs  and  may  require  several  weeks,  at  which  time  some  dated  searches  may  no  longer  be  relevant 

To  improve  this  situation,  ihe  automated  access  and  post-processing  capabilities  of  the  LLNL  Tech¬ 
nology  Information  System  (TIS)  were  tested  by  the  NIAC  center  at  the  University  of  Southern  California. 
Their  procedures  and  results  are  described  later  in  this  report. 


2.4  Federal  Laboratory  Consortium  (FLC) 

A  laboratory  consortium  for  technology  transfer  was  formally  begun  in  1971  to  facilitate  the  effective 
use  of  new  products  and  services  derived  from  research  sponsored  by  the  Department  of  Defense.  At  that 
time,  the  National  Science  Foundation  established  a  technology  transfer  liaison  service  to  promote  com¬ 
munication  between  DOD  laboratories  and  other  agencies.  The  resulting  DOD  Consortium  was  estab¬ 
lished  to  improve  laboratory  communication  and  to  expand  the  secondary  utilization  of  laboratory-devel¬ 
oped  technology.  Because  of  the  interest  in  technology  transfer  expressed  by  other  federal  laboratories, 
the  DOD  Consortium  was  enlarged  in  1974  to  a  federal  laboratory  consortium,  and  the  FLC  was  estab¬ 
lished.  Membership  in  the  FLC  is  open  to  all  federal  laboratories.  Today,  the  FLC  has  a  current  member¬ 
ship  of  over  200  R&D  laboratories  and  centers  from  11  federal  agencies.7  This  growth  in  membership  | 

indicates  how  interested  the  laboratories  and  their  agencies  are  in  maximizing  opportunities  for  new  | 

applications  of  innovative  R&D.  I 


2.5  Technology  Transfer  (T2)  Society 

The  T2  Society  was  incorporated  in  1975.8  The  founders  felt  there  was  a  need  for  a  nonprofit  interna¬ 
tional  organization  that  could  act  as  a  catalyst  for  accelerating  the  movement  of  technology  to  its  ultimate 
application. 

The  Society  has  grown  steadily,  attracting  professionals  across  the  U.S.  and  in  foreign  countries.  Its 
membership  includes  individuals  from  industry,  government,  research,  and  education  who  are  concerned 
w  ith  issues  of  technology  transfer,  utilization,  assessment,  and  forecasting.  The  Society  publishes  a  news¬ 
letter  and  conducts  an  annual  international  conference.  Together  with  other  societies,  it  has  also  co¬ 
sponsored  a  number  of  seminars  on  advanced  technology.  The  Journal  of  Technology  Transfer,  a  peer 
journal  for  the  presentation  of  papers  describing  experiences,  processes,  methods,  views,  and  research 
findings  is  sent  free  to  all  members.  A  library  on  the  transfer,  utilization,  assessment,  and  forecasting  of 

5 


ItHMIilttBMMMMiaMMiaiMMM T  — - - - -  - - — — 


technology  is  operated  at  the  Society's  headquarters  in  Los  Angeles.  It  is  the  repository  for  all  information 
generated  by  the  Society  and  its  memK-rs. 

The  Society  is  empowe  d  to  enter  into  contracts  for  studies  and  other  work  reused  to  its  mission.  The 
Productivity  Improvement  Council,  created  by  the  Society  to  address  the  growing  national  problem  of 
declining  productivity,  is  an  example  of  such  work.  The  Society  is  a  nonprofit  organization.  Activities  of 
the  Society  are  supported  by  individual  members  and  corporate  du 's. 


2.6  Irteragency  Information  Exchange  (HE)  Group 

This  recent  effort  to  promote  the  sharing  of  advanced  information  technology  among  agencies  was 
reconstituted  in  the  spring  of  1980.  The  Group,  which  consists  of  1  to  2  representatives  from  17  federal 
agencies  and  organizations,  was  organized  to  encourage  the  use  of  information  svtems  that  have  govern¬ 
ment-wide  applicability  and  to  facilitate  the  development  of  applications  that  have  significant  utility 
across  departmental  and  agency  lines. 

The  need  for  such  a  group  was  identified  dramatically  in  the  Federal  Data  Processing  Reorganization 
Report  (FDR).  The  thrust  of  the  recommendations  called  for  the  establishment  of  a  strong  focal  point 
within  the  Executive  Branch  that  would  "encourage  and  facilitate"  the  interchange  and  interagency  utili¬ 
zation  of  common,  beneficial  applications,  as  well  as  provide  a  positive  force  for  the  introduction  of  new 
and  advanced  information  technology  in  the  federal  government.9 

The  HE  recognized  in  particular  the  difficulty  of  accessing  and  utilizing  available  information  re¬ 
sources.  To  solve  the  problem  of  dealing  with  the  variety  of  information  systems,  incompatible  computer 
resources,  and  complex  and  different  access  procedures,  the  Group  selected,  in  1980,  the  T1S  at  LLNL  for 
its  electronic  mail  communications  among  group  membeis  and  for  automated  exploratory  networking 
with  federal  information  centers.  Of  particular  interest  to  the  HE  are  the  TIS  transparent  access  procedures 
and  post-processing  techniques  for  text  and  data. 

2.7  Department  of  Energy  Implementation 
of  PL-96-480 

The  Department  of  Energy  implemented  PL-96-480  by  DOE  Order  5800.1,  issued  by  William  S. 
Heffelfinger,  Assistant  Secretary  for  Management  and  Administration,  March  25,  1982.  The  nature  and 
policy  of  the  Order  are: 

•  The  laboratory's  explicit  technology  transfer  activity  will  be  funded  from  overhead. 

•  The  laboratory's  explicit  technology  transfer  activity  will  be  placed  organi  ationally  so  as  to  have 
labwide  support  and  effective  access  to  all  program  activity. 

•  The  laboratory  will  include  a  five-year  technology  transfer  plan  as  part  of  the  institutional  plan, 
showing  level  of  effort  and  scope  of  activity. 

•  The  laboratory  will  prepare  an  annual  report  describing  significant  technology  transfer  achieve¬ 
ments  over  the  previous  fiscal  year. 

The  law  mandates  at  least  0.5%  of  agency  R&D  funding  for  technology  transfer.  Each  agency  and 
laboratory  has  implicit  programs  that  comply  in  part  with  this  mandate.  However,  the  establishment  of 
explicit  local  offices  at  the  laboratories  to  interact  with  the  ongoing  programs  and  act  as  catalysts  for  the 
technology  transfer  process  is  new. 
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2.8  LLNL  Office  of  Research  and 
Technology  Applications  (ORTA) 

The  LLNL/ORTA  office  was  established  in  1981.10  Both  PL-96-480  and  Order  5800.1  clarified  a 
number  of  the  technology  transfer  policies  that  varied  greatly  among  individual  laboratories  and  provided 
specific  guidelines  for  implementation.  The  ORTA  office  at  LLNL  is  a  part  of  the  Laboratory's  mission.  Its 
goals  and  objectives  are: 

•  To  provide  maximum  benefit  to  LLNL  programs,  consistent  with  the  intent  of  the  law. 

•  To  document  and  plan  present  work  to  demonstrate  that  much  of  it  qualifies  as  technology 
transfer. 

•  To  structure  activities  so  that  program  offices  have  the  dominant  role  in  technology  transfer. 

•  To  coordinate  and  support  technology  transfer  through  the  ORTA,  not  by  control  of  program 
activity. 

•  To  make  maximum  use  of  contract  personnel  for  specialized  jobs  not  generally  needed  by  the 
Laboratory. 

The  above  objectives  are  interrelated  and  can  be  viewed  as  an  ongoing  process  linking  the  activities 
of  several  LLNL  programs.  The  LLNL/ORTA  is  charged  with  administering  these  efforts  and  with  stimu¬ 
lating  and  fostering  an  awareness  throughout  the  Laboratory  of  both  its  obligation  and  the  benefits  of 
technology  transfer.  Recipients  of  the  technologies  transferred  include  firms  in  the  industrial  and  commer¬ 
cial  sectors,  state  and  local  governments,  and  related  institutions  and  entities. 

One  of  the  activities,  application  assessment,  is  the  task  of  evaluating  the  potential  of  all  R&D  projects 
and  searching  for  new  and  secondary  uses.  Program  personnel  are  asked  to  make  the  initial  selection  of 
specific  transfer  candidates.  Final  selection  will  be  made  by  a  technical  committee  under  ORTA  staff 
supervision.  Technical  assistance  will  generally  be  provided  at  no  cost  to  local  governments.  Direct  ex¬ 
penses  for  technology  assistance  will  normally  he  paid  by  industry.  A  technical  steering  committee  is 
helping  the  ORTA  implement  the  LLNL  technology  transfer  program. 


LLNL  Technology  Transfer  Frogram 


I 

i  o  PLANNED 
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3.  TIS  Technology  Information  System 

The  mandated  objectives  of  technology  transfer  require  timely  identification,  reliable  assessment,  and 
effective  communication  of  R&D  results  among  agencies,  members  of  the  FLC,  the  T2  Society,  and  local 
laboratory  offices. 

The  operational  capabilities  of  the  TIS  meet  these  requirements  and  permit  the  creation  of  a  unified 
view  of  available  resource.,  .or  the  technologist.  Unlike  other  information  systems,  TIS  gives  authorized 
viewers  direct  automated  access  to  geographically  distributed  information  centers,  as  well  as  permitting 
them  to  extract,  aggregate,  and  analyze  descriptive  and  numeric  data  in  an  interactive,  self-guided  manner 
and  to  quickly  transfer  the  desired  results  to  the  end-user.  Moreover,  iequestors  can  link  their  terminals  to 
those  of  information  specialists  and,  from  their  offices  or  homes,  watch  the  progress  of  a  search  on  their 
own  video  screens;  they  can  also  discuss  its  applicability  with  experts  simultaneously  by  telephone. 
Relevant  results  can  then  be  post-processed  into  suitable  lookup  tables  and  printed  immediately  at  the 
requestor's  location.  The  flexible  database  management  capabilities  of  TIS,  its  direct  interconnection  with 
electronic  word  processors  and  typesetters,  and  electronic  mail  provide  a  work  environmen,  well  suited 
for  the  logging  of  requests  for  technical  infoimation  and  for  recording  and/or  reporting  of  ORTA 
responses. 

Under  development  and  in  operation  since  1975,  TIS  can  be  accessed  from  standard  computer  termi¬ 
nals  by  telephone  dial-up,  WATS  lines,  TYMNET,  and  the  extensive  ARPA  computer  network.1112 

These  TIS  capabilities  and  communication  links  are  highly  effective  when  applied  to  searches  for 
advanced  applications  of  known  technological  needs.  More  significantly,  post  processing  tools  under 
development  on  TIS  permit  the  tentative  identification  of  technological  breakthroughs  and  innovations 
that  may  not  be  obvious  when  examined  by  conventional  means. 


3.1  TIS  — An  Intelligent  Gateway  to 
Technological  Resources 

The  automated  and  transparent  access  procedures  to  different  information  centers  are  part  of  our 
prototyping  of  an  Intelligent  Gateway  Computer  (1GC).  Users  of  TIS  may  consult  the  availability  of 
programmatic  technical  resources  stored  internally  for  their  use,  or  made  available  to  them  via  TIS  on 
external  information  sources; 


The  Technology  Information  System  "IIS" 
An  Intelligent  Gateway  Computer 


"One  does  not  build  a  library  by  writing  books! 


"likewise,  to  build  Information  Centers,  we  must 
encourage  the  collection  and  analysis  of  available 
data,  by  proven  techniques  and  worldwide 
communications,  for  the  rapid  transfer  of 
knowledee." 


Internal  Information  Sources 
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•  Directory  to  resources 

•  Programmatic  database 

•  Personal  computer/database 


Each  external  information  center  is  qualified  online  on  TIS  by  its  accreditation,  the  availability  and 
cost  of  its  databases,  an  annotated  description  of  salient  commands,  and  prevailing  up-times.  This  in¬ 
formation  is  extracted  by  periodic  transfer  from  extended  information  centers  to  TIS  for  online  consulta¬ 
tion  by  the  TIS  user  community  prior  to  use,  thereby  saving  time  and  communication  costs.  Where 
appropriate,  access  to  external  information  resources  is  granted  to  TIS  users  on  an  individual  basis  by  the 
Database  Administrator.  Users  gain  access  to  othei  information  centers  simply  by  giving  the  command 
CONNECT,  followed  by  the  target  name  of  the  desired  resource:  e.g.,  ‘CONNECT  DOE/RECON12  will 
establish  access  to  DOE/RECON  at  1200  baud. 

Alternatively,  users  may  specify  the  TIS  option  number  of  the  desired  resource,  which  is  part  of  each 
online  menu.  In  either  case,  users  need  not  be  familiar  with  telephone  dial-up  numbers,  accounts,  pass¬ 
words,  or  peculiar  protocols. 

User  communities  of  the  TIS  establish  their  own  views  of  internal  and  external  programmatic  re¬ 
sources,  in  a  self-guided  manner,  without  programmer  intervention.  Database  Administrators  assigned  to 
each  user  community  control  access  rights  on  a  need-to-know  basis.  Individual  users  see  only  those 
resources  (e.g.,  datafiles,  interactive  models  graphs,  and  reports)  to  which  they  have  access.  An  exception 
is  the  external  resources  we  advertise  to  promote  their  use.  When  a  use’  is  finished  using  a  TIS-provided 
external  resource,  his  access  rights  can  be  removed  by  the  TiS  Database  Administrator,  and  no  change  of 
passwords  is  required  since  none  were  disclosed. 

The  DIAL  command  provides  an  equally  powerful,  but  user-controlled,  method  for  accessing  other 
information  centers  and  computers.  In  this  case,  the  user  is  prompted  to  specify  the  telephone  number, 
baud  rate,  parity,  and  other  parameters.  TIS  then  establishes  the  communication  similar  to  an  automated 
telephone  dialer.  Here  users  have  to  provide  their  own  accounts  and  passwords  for  log-in  on  the  external 
host  machine.  Such  procedures  can  be  saved  for  personal,  routine  use. 

When  an  account  with  another  information  center  is  opened  for  TIS.  the  vendor  bills  TIS  at  LLNL, 
which,  in  turn,  deducts  the  appropriate  ccsts  from  the  responsible  programmatic  accounts.  When  users 
establish  their  own  accounts  with  information  centers  and  use  them  via  TIS,  they  are  billed  directly  by  the 
vendors,  who  cannot  distinguish  by  what  means  the  user  accessed  and  used  their  information  center.13 

For  the  LLNL/ORTA  technology  transfer  program,  we  established  automated  access  to  the  TECTRA 
information  system  at  the  School  of  Business  and  Public  Administration  at  the  California  State  University 
in  Sacramento,14  the  Agnew  Tech-Tran  Inc.  online  translation  service,,11'  and  the  DOE  patent  database  at 
the  DOE/RECON  information  center  in  Oak  Ridge,  Tennessee.  Some  of  these  files  were  transferred  by 
electronic  means  to  TIS  and  installed  online  for  enhanced  use  by  a  wider  user  community.  They  include; 

•  International  Technology  Transfer  Directory  of  more  than  2200  experts. 

•  TECTRA  newsletter  guide  to  about  300  transferable  technologies. 

'*  Online  address  list  to  230  federal  laboratories. 

•  Subject  index  to  more  than  2000  DOE  patents  issued  1975-1982. 

Automated  procedures  to  access  the  technology  transfer  files  of  other  agencies  are  in  preparation. 
Individually  authorized  users  of  TIS  may  search  the  bibliographic  information  centers  of  DOE,  DOD,  and 
NASA  for  applicable  technologies.  The  Chemical  Information  System  (CIS)  of  NIH/EPA,  the  DARC 
system  in  France,  and  the  DECHEMA  thermophysical  properties  system  in  Germany,  among  others,  may 
be  consulted  online  for  chemical  and  toxicological  information. 

The  TIS  Intelligent  Gateway  gives  users  a  unified  view  of  available  resources  without  obliging  indi¬ 
vidual  information  centers  to  conform  to  standard  command  languages  or  formats.  This  has  the  added 
benefit  of  inadvertently  normalizing  dissimilar  procedures  at  different  information  centers  whose  staffs 
are  encouraged  to  view  the  rendition  of  their  system  via  TIS,  to  test  the  capabilities  of  others,  and  to  adopt 
those  capabilities  that  are  superior  to  their  own. 
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3.2  Downloading  of  Relevant  Information 

Most  online  bibliographic  information  is  still  being  sold  by  offline  printing  following  a  search.  At 
best,  the  citations  are  shown  or  pi  in  ted  in  chronologically  reverse  order — last  publication  first.  When  a 
comprehensive  and  retrospective  search  is  carried  out,  the  end-user  is  faced  with  piles  of  printouts  con¬ 
taining  redundant  citations,  in  different  formats,  from  different  vendors.  Usually,  the  returns  have  no 
indexes  or  contents  lists,  and  manual  review  and  organization  of  the  material  is  required.  Much  of  the 
information  thus  received  is  probably  being  discarded  unused. 

To  eliminate  some  of  these  difficulties,  the  user  of  TIS  can  initiate  extraction  and  downloading  of 
information  from  another  system  to  TIS  in  two  ways:  First,  the  SAVEON  option  permits  extraction  of 
information  when  used  with  the  CONNECT  command  discussed  previously.  In  this  case,  all  information 
seen  on  the  screen  during  one  session  is  placed  into  a  user-named  file. 

Post-Processing  of  Bibliographic  Information 


Conventional 


1970  1980 


Look  at  it. 


TIS  Procedure: 


10 


Approximately  100  citations  with  abstracts  can  be  extracted  and  saved  in  10  minutes  at  1200  baud  by 
asynchronous  telephone  dial-up.  Faster  transmission  is  possible  with  9600-baud  dedicated  and  condi¬ 
tioned  synchronous  lines.  Second,  the  DIAL  command  permits  extraction  and  downloading  into  one  or 
more  individual,  user-named  files  that  can  be  opened  and  closed  at  liberty  by  special  control  characters 
during  a  session,  e.g. 

ESCAPE  CTRL-A — prompts  the  user  for  a  file  name  and  saves  the  viewed  information  therein.  An 
additional  ESC-CTRL-A  closes  the  file.  If  the  file  already  exists,  the  new  information  is  appended  to 
permit  progressive  creation  of  a  cumulative  subject  datafile. 

ESCAPE  CTRL-B — sends  a  local  file  from  TIS  to  a  remote  machine.  This  has  particular  importance  when 
downloaded  and  saved  information  is  to  be  transferred  to  more  powerful  computers  for  analysis,  cr  is  to 
be  shared  with  someone  else  via  TIS. 

Other  special  control  characters  permit  the  user  to  stop  the  viewing  and/or  saving  of  information  and 
to  address  the  local  or  remote  computer  selectively. 

The  legal  and  monetary  implications  of  downloading  and  sharing  information  extracted  from  other 
centers  must  be  considered  carefully. 


3.3  Post-Processing  of  Bibliographic 
Information  Online 

When  a  retrospective  search  is  conducted  for  a  new  field  of  interdisciplinary  research,  it  is  not 
unusual  to  obtain  thousands  of  citations  from  different  information  vendors,  in  different  formats,  with 
redundant  overlap. 

Recently,  a  request  made  to  the  Transportation  Systems  Research  program  at  LLNL  to  identify  for¬ 
eign  R&D  in  electric  batteries  yielded  1.5  ft  cf  printouts  from  federal  and  commercial  information  vendors. 
The  2000  citations,  obtained  by  conventional  means  at  a  cost  of  about  $1700,  were  quite  useless.  It  was 
difficult  to  convey,  in  time,  meaningful  statistics,  insights,  or  magnitudes  of  ongoing  R&D  abroad  to  DOE 
headquarters.  The  offline  prints  alone  took  six  days  to  arrive  by  mail.  One  solution  to  this  problem,  at 
past  for  databases  from  DOE/RECON,  is  the  post-processing  of  downloaded  citations  with  1  IS. 

First,  simple  TIS  commmands  save  the  retrieved  information  in  a  convenient,  user-defined  filing 
system.  This  permits  results  to  be  organized  and  aggregated  in  a  suitable  manner.  Redundant  citations  can 
be  eliminated  by  their  congruent  main  data  fields,  primarily  by  compa-ison  of  authors  and  titles.  Second, 
the  resulting  unique  set  is  then  reviewed  and  analyzed  online  by  self-guided,  post-processing  routines. 

•REVIEW — Citations  for  relevancy. 

•DISPLAY — Graphs  of  publication  rates. 

•PERMUTE— Multiterm  expressions  in  data  fields. 

•CROSS-CORRELATE— Contents  of  data  fields,  with  statistics. 

•CONCORD — Citations  by  author,  subject,  descriptors,  etc. 

The  REVIEW  command  permits  online  determination  of  relevancy.  Citations  are  shown  on  the 
screen,  reformatted  by  accentuation  and  indentation  of  titles,  authors,  and  abstracts.  This  renders  them 
more  readable  than  citations  commonly  offered  by  information  centers.  The  viewer  may  keep  or  discard 
any  citation  shown  and  assign  his  own  category  and  relevancy  cede.  He  may  add  comments,  order  the 
ful. -length  text,  and  define  and  fill  new  data  fields  for  numeric  and/or  administrative  purposes.  Retained 
and  annotated  citations  are  saved  in  new  user-named  files.  Fields  defined  during  the  review  process  can 
subsequently  be  used  with  other  fields  for  post-processing. 


s 


Online  Determination  of  Relevant  Set 


The  DISPLAY  of  the  publication  rate  for  a  particular  search  topic,  institute,  or  author  immediately 
indicates  the  effort  or  growth  in  the  field  of  activity  and  might  well  be  the  first  display  an  end-user  wishes 
to  see.  In  most  cases,  an  apparent  decline  follows  a  sharp  rise  in  the  publication  rate.  This  decline  is 
predominantly  the  time  lag  between  the  publication  of  the  primary  literature  and  its  entry  into  the 
secondary  online  database  holdings.  To  appreciate  the  actual  decrease  or  increase  in  publications  in  a 
particular  field,  one  has  to  compare  it  with  the  total  annual  increase  in  the  publication  rate.  In  the  sciences, 
this  rate  is  now  about  13%.16 


Preliminary  analysis  of  patents  issued  to  the  former  Atomic  Energy  Agency,  the  former  Energy 
Research  and  Development  Administration,  and  the  present  Department  of  Energy  shows  yet  unex¬ 
plained  irregularities  in  recent  years  and  may  signal  a  decline.  The  decline  may  be  the  result  of  decreased 
R&D  funding  in  science  and  technology  in  the  1970s. 


The  most  significant  uspeci  of  post-processing  is  probably  the  time-dependent  change  or  momentum 
of  a  particular  field  in  R&D,  derived  from  the  statistics  of  its  permuted  descriptive  terms,  the  PERMUTE 
command  of  our  post-processing  routines  provides  this  option  by  counting  the  number  of  times  a  speci¬ 
fied  term  appears  in  the  message-carrying  fields  of  citations,  like  the  title,  abstract,  descriptor,  category, 
etc.  This  is  done  by  analyzing  single  and  compound  expressions  containing  up  to  four  terms  (e.g.,  solar 
energy  conversion  experiments).  All  compound  expressions  of  this  type  appearing  in  the  selected  data 
fields  are  presented  to  the  viewer  online,  alphabetically  ordered,  with  their  frequency  of  occurrence.  The 
following  table  shows  this  for  the  leading  descriptors  of  the  2000  most  recent  DOE  patents.'7 


842  DESIGN 
2SE  FABR1 CATION 
94  SPECIFICATIONS 
8  3  LMF8P  Ty  ‘  PE  PC  TOPS 
81  ELECTRODES 

77  RADIOACTIVE  WASTE  PROCESSING 

67  OPERATION 

66  CHEMICAL  PREPARATION 

66  HIGH  TEMPERATURE 

6b  PRODUCTION 

64  EQUIPMENT 

64  GASES 

63  WATER 

52  LASERS 

49  COAL 

49  METALS 

49  SEALS 

49  TUBES 

47  REMOVAL 

47  REPROCESSING 


46  HYDROGEN 
44  ELECTRONIC  CIRCUITS 
44  SEPARATION  PROCESSES 
43  AQUEOUS  SOLUTIONS 
43  CHEMICRL  COMPOSITION 
42  CHEMICAL  REACTIONS 
40  CERAMICS 
39  VALVES 

36  CARBON  DIOXIDE  LASERS 

38  COAl  GASIFICATION 
3e  CONTAINERS 

39  laser  radiation 
38  LIQUID  WASTES 

36  MICROSPHERES 

37  CAThDDES 

37  LASER  ISOTOPE  SEPARATION 
36  ALUMINIUM  ALLOYS 
36  CONTROL  systems 
36  FLUIDIZED  BED 
36  HYDROGEN  PRODUCTION 


CROSS-CORRELATIONS  of  expressions  between  any  two  fields  of  the  citations  can  provide  new 
insight.  For  example,  by  cross-correlating  authors,  we  can  see  at  a  glance  who  is  working  with  whom.  A 
cross-correlation  of  the  author  field  with  the  descriptor  field  shows,  in  alphabetic  order,,  the  statistics  of 
indexing  terms  assigned  to  the  work  of  a  particular  person  for  his  entire  professional  career.  When  carried 
out  in  yearly  increments,  this  routine  can  be  used  to  judge  the  change  of  emphasis  with  time. 
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CONCORDANCES  generated  by  author,  descriptor,  corporated  author,  or  country  yield  succinct 
listings  of  bibliographic  citations  in  a  particular  field.  These  alphabetical  indexes  are  similar  to  those 
commonly  produced  as  lookup  tables  for  authors  or  subjects.  In  this  case,  they  are  created  at  the  pleasure 
of  the  user,  online,  on  the  contents  of  any  citation  field.  The  following  table  shows  an  extract  from  our 
forthcoming  concordance  of  DOE  patents.  It  will  provide  a  convenient  ’1st  of  patents  available  for  licens¬ 
ing  in  different  fields  of  technology. 

Extracts  from  Concordances  of  DOE  Patents  by  Author  and  Descriptor. 


If <1*1 ,  8  R 

1Q7S  Grant.  T  H. . 

B  ft  .  O'Brien,  H  A 

1976  G.ant.  P.M  .  Erd«l, 
B.R  .  0‘Brien,  H  A 

EsstbAqqcrs,  j 

1975  tsscbagqer$.  J 


1976  Ev« nstf  K  W 

tvellelSh,  J  h 

1976  Sartor/.  W  k.; 
Eveilelgh.  J.M. 


RUBIDIUM  RADIOISOTOPE 
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1975  Farnum.  E  H 
fries,  R.J 


1976  fcrntm,  E  N 
Fries,  R.J. 


Farrlor,  W  L  Jr 

1976  Farrlor,  U.l.  Jr. 


RADIOISOTOPE  77POC19S44 


ENERGY  ABSORBER  FOR 
SOOIUK-HtATED  HEAT 
EXCHANGER 


PROTECTIVE  AIR  LOCK 


CENTRICUGE  APPARATUS 


METHOD  TOR  SUING  77P0040417 

HOLLOW  MICROSPHERES 

METHOD  FOR  NONDESTRUCT -  77P0085U? 
!VE  FUEL  ESSAY  OF  LASER 
FUSION  TARGETS 


REGENERABLE  SORBENT  AND  77P0O85122 
METHOD  FOR  REMOVING 
HYDROGEN  SULFIDE  FROM 
HOT  GASEOUS  Minium 


COAL  DEPOSITS 

1976  Archibald,  P.B. 


1976  Fisher,  S.T.; 
Fisher,  C  B. 
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GASIFICATION  OF  CARBON¬ 
ACEOUS  SOLIDS 


GASIFICATION  OF  CARBON-  77P008S18S 

ACEOUS  SOLIDS 

NETHOO  AND  APPARATUS  FOR  76P0067836 

REMOVING  COARSE  UNEHTRAINEO 

CHAR  PARTICLES  FROM  THE 

SECOND  STAGE  OF  A  TUO- 

STAGr.  COAl  GASIFIER 

EXTRACTION  OF  HYDROCARBONS  77P0005I29 

IN  SITU  FROM  UNDERGROUND 
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NETHOO  TOR  CONTROL  OF  SUB-  77P00B5I89 

SURFACE  COAl  GASIFICATION 

OXIDATION  OF  COAl  MATER  SEURRT  77P0005171 
FEED  TO  HYDROGASIFIER 


Output  from  post-processing  routines  can  be  saved  in  files  for  subsequent  use  or  tor  transfer  to 
electronic  word  processors  and  merging  with  reports,  or  it  can  be  sent  to  typesetters  as  camera-ready  copy 
for  publication.  It  can  also  serve  as  input  to  online  data  files  for  interactive  use.  A  description  of  present 
TIS  post-processing  capabilities  has  been  published.19 

These  post-processing  and  text  analysis  techniques  have  a  direct  bearing  on  technology  transfer 
programs  by  packaging  their  descriptive  information  in  a  comprehensive,  well-organized  manner  for 
quick  lookup.  For  those  without  direct  access  to  TIS,  we  generate  the  information  by  machine  for  printing 
or  for  use  as  microfiche.  For  example,  the  descriptive  concordance  of  2000  patents  issued  to  DOE  and  its 
predecessors  since  1975  amounts  to  an  alphabetic  index  of  3300  descriptive  terms  from  the  DOE/RECON/ 
INIS  thesaurus  on  1000  pages  of  print,  or  about  12  microfiche  with  a  24  X  reduction.18 


3.4  Finding  Pearls  in  the  Mire 

Until  now  we  have  discussed  the  rendition  of  technical  information  in  response  to  desired  advanced 
applications  in  known  fields  of  interest.  Of  potentially  much  greater  significance  is  the  filtering  of  the 
large  volume  of  technical  information  to  expose  those  citations  that  may  signify  potential  breakthrough 
and  discovery.  This  will  become  crucial  as  we  approach  the  1980s  and  are  faced  with  ever  larger  volumes 
of  factual  data  each  year.  To  increase  our  national  productivity,  we  must  establish  procedures  to  filter  the 
data  and  to  recognize  creative  publications  quickly. 

One  approach  is  to  filter  out  those  publications  in  a  general  technological  category  that  deviate  from 
previous  norms  by  inspecting  the  single  and  multiterm  expressions  used  by  authors  in  titles  and  abstracts. 
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and/or  the  use  of  descriptors  by  indexers.  This  type  of  text  analysis,  although  not  new,  has  some  fascina¬ 
tion  because  it  permits  us  to  recognize  significant  migration  or  cross-fertilization  of  new  ideas  in  R&D 
work. 

Take  the  laser  field,  for  example.  When  one  creates  an  authority  list  of  all  single  and  multiterm 
expressions  derived  from  titles  and  abstracts,  one  arrives  at  a  reasonably  stable,  slowly  changing  body  of 
descriptive  terms.  Newly  appearing  terms  can  easily  be  set  aside  and  used  to  mark  citations  for  closer 
inspection.  It  is  thus  possible  to  find  the  citation  where  laser  beam*  were  first  used  to  weld  the  retina  in  an 
eye.  In  other  words,  one  can  filter  out  those  citations  that  somehow  do  not  follow  the  common  pattern  of 
previous  descriptive  indexing  or  word  usage.  They  contain  either  typographical  errors  or,  potentirlly, 
technological  pearls.  Other  examples  quickly  come  to  mind.  The  techniques  for  doing  this  type  of  analysis 
have  been  known  since  the  time  inverted  tables,  or  secondary  indexes,  were  introduced  for  the  machine- 
aided  lookup  of  facts.  They  could  be  used  in  the  1980s  as  a  new  means  of  SDI  service,  signaling  unusual 
and  different  publications,  or  discoveiy  ready  for  advanced  applications. 

The  extraction  of  intelligence  from  the  literature  in  science  and  technology  and  its  rapid  application  in 
industry  is  expected,  if  adequately  supported,  to  have  far-reaching  effects  on  our  economy.  The  recent 
impetus  given  to  the  technology  transfer  program  as  a  nationally  mandated  activity  will  hopefully  permit 
the  development  of  advanced  pattern-recognition  techniques  for  text  and  data  analysis. 


3.5  Linking  the  Technology  Requestor 
with  the  Expert 

Users  of  the  TIS  Intelligent  Gateway  can  link  their  terminals  to  those  of  others  logged  in  on  the 
machine.  Requestors  of  information  may  thus  work  jointly  from  the>r  terminals  with  experts  while  search¬ 
ing  for  technological  information  on  a  distant  information  system  in  this  country  or  abroad 

The  effectiveness  of  this  TIS  procedure  was  demonstrated  by  the  NASA  Industrial  Applications 
Center  (NIAC)  at  the  University  of  Southern  California  earlier  this  year  when  their  information  specialists 
conducted  searches  on  DOE/RECON,  NASA/RECON,  and  other  information  centers  by  joint  viewing 
and  telephone  discussion  of  results  with  clients  elsewhere  in  the  country.20  These  tests  were  made  in 


Simultaneous  graphical  display  and  voice  communication! 
Terminals  of  expert  and  student(s)  aie  equivalent! 


preparation  for  more-extensive  evaluations  planned  under  the  auspices  of  the  Small  Business  Administra¬ 
tion  with  NASA-NIAC-USC  and  clients  in  Idaho.  The  immediate  post-processing  of  retrieved  results  and 


the  printing  of  relevant  information  at  the  clients'  terminals  was  shown  to  save  time  and  money.  This 
methodology  can  also  be  used  for  nationwide  tutorials  and  class  instruction.21 


In  a  transcontinental  demonstration  last  Decemeber,  Dr.  C.  Sullivan,  via  T1S,  gave  a  tutorial  on  the  :  u 

use  of  the  Chemical  Information  System  to  a  class  of  48  professionals  at  the  University  of  Ouro  Preto,  )  | 

Brazil,  without  leaving  his  desk  in  Washington,  D.C.  Participants  in  this  training  couse  on  database  \ 

management  in  the  geosciences,  under  sponsorship  of  the  United  National  General  Information  Program  -  \ 

(UNESCO/PG1),  heard  his  voice  and  saw  the  search  strategies  and  chemical  structures  projected  on  the  •  \ 

screen  of  their  auditorium.  They  could  ask  questions  and  try  new  skills  under  Dr.  Sullivan's  guidance.  j 

Arrangements  for  this  tutorial  had  been  made  the  day  before  from  Brazil.21  ;  1 


3.6  The  Intelligent  Gateway  Computer  as  an 
Intermediary  to  Technical  Information 

Audio-visual  linking  of  users  across  nations  and  continents  suggests,  in  a  general  sense,  a  new 
application  of  the  TIS  Intelligent  Gateway  Computer  (ICG)  as  an  electronic  intermediary  between  request¬ 
ors  and  vendors  of  technological  information.  Another  decade  may  go  by  before  administrators  feel 
comfortable  using  their  own  computer  terminals.  Even  then,  it  will  probably  be  more  efficient  for  the 
infrequent  computer  user  to  unburden  his  mind  with  requests  for  technical  information  by  asking  a  staff 
assistant  or  secretary  to  send  the  requests  by  electronic  mail  or,  even  better,  to  dictate  them  directly  into  a 
message  machine. 

We  have  implemented  these  capabilities  on  the  IGC  prototype  by  making  the  telephone  number  for 


returned  to  the  requestor's  ICG  mailbox,  for  joint  discussion  with  the  expert  if  necessary  or  for  immediate 
printing  at  the  requestor's  facility.  The  TIS  Intelligent  Gateway  Computer  will  thus  become  the  electronic 
intermediary  between  the  end-user  and  the  information  expert.23  24 


Conclusion 

Advanced  capabilities  in  information  management  and  communications,  under  development  for  the 
!GC  on  the  TIS  at  LLNL,  are  being  used  to  establish  a  focal  point  for  the  technology  transfer  community. 
Dissimilar  and  geographically  distributed  technological  resources  can  thus  be  viewed  and  used  as  an 
integrated  system  and  provide  a  cost-effective  means  for  the  exchange  and  sharing  of  information  among 
federal  laboratories  and  technologists  at  universities  and  in  industry.  This  is  expected  to  contribute  to 
successful  implementation  of  the  Stevenson-Wydler  Technology  Innovation  Act  and  to  enhance  pro¬ 
ductivity  in  the  United  States. 

Concerns  have  been  expressed  that  the  collection  and  persuasive  presentation  of  well-organized 
technical  information  could  also  be  acquired  by  other  countries.  Its  use  by  developing  countries,  who 
struggle  in  the  marketplace  for  their  fair  share  of  rewards,  should  prove  to  be  mutually  beneficial.  The 
situation  is  different  when  we  question  the  transborder  flow  of  advanced  technologies  to  countries  who 
compete  with  us  both  economically  and  politically.  The  Information  Science  and  Technology  Act  (H.R. 
313726)  and  Section  38A  of  the  Control  of  Export  of  Arms  Information  address  themselves  to  both  issues. 

A  definitive,  practical  policy  for  the  role  of  government  in  the  evaluation  and  dissemination  of 
numeric  data  in  science  and  technology  is  much  needed.  Whatever  the  outcome,  it  is  of  paramount 
importance  that  we  learn  to  be  the  first  to  apply  the  technology  we  develop. 
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